Tie-respecting bootstrap methods for estimating distributions of sets and functions of eigenvalues

Bootstrap methods are widely used for distribution estimation, although in some problems they are applicable only with difficulty. A case in point is that of estimating the distributions of eigenvalue estimators, or of functions of those estimators, when one or more of the true eigenvalues are tied. The $m$-out-of-$n$ bootstrap can be used to deal with problems of this general type, but it is very sensitive to the choice of $m$. In this paper we propose a new approach, where a tie diagnostic is used to determine the locations of ties, and parameter estimates are adjusted accordingly. Our tie diagnostic is governed by a probability level, $\beta$, which in principle is an analogue of $m$ in the $m$-out-of-$n$ bootstrap. However, the tie-respecting bootstrap (TRB) is remarkably robust against the choice of $\beta$. This makes the TRB significantly more attractive than the $m$-out-of-$n$ bootstrap, where the value of $m$ has substantial influence on the final result. The TRB can be used very generally; for example, to test hypotheses about, or construct confidence regions for, the proportion of variability explained by a set of principal components. It is suitable for both finite-dimensional data and functional data.Comment: Published in at http://dx.doi.org/10.3150/08-BEJ154 the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm

Prediction of Traffic Complexity and Controller Workload in Mixed Equipage NextGen Environments

Controller workload is a key factor in limiting en route air traffic capacity. Past efforts to quantify and predict workload have resulted in identifying objective metrics that correlate well with subjective workload ratings during current air traffic control operations. Although these metrics provide a reasonable statistical fit to existing data, they do not provide a good mechanism for estimating controller workload for future air traffic concepts and environments that make different assumptions about automation, enabling technologies, and controller tasks. One such future environment is characterized by en route airspace with a mixture of aircraft equipped with and without Data Communications (Data Comm). In this environment, aircraft with Data Comm will impact controller workload less than aircraft requiring voice communication, altering the close correlation between aircraft count and controller workload that exists in current air traffic operations. This paper outlines a new trajectory-based complexity (TBX) calculation that was presented to controllers during a human-in-the-loop simulation. The results showed that TBX accurately estimated the workload in a mixed Data Comm equipage environment and the resulting complexity values were understood and readily interpreted by the controllers. The complexity was represented as a "modified aircraft account" that weighted different complexity factors and summed them in such a way that the controllers could effectively treat them as aircraft count. The factors were also relatively easy to tune without an extensive data set. The results showed that the TBX approach is well suited for presenting traffic complexity in future air traffic environments

Trajectory-Based Complexity (TBX): A Modified Aircraft Count to Predict Sector Complexity During Trajectory-Based Operations

In this paper we introduce a new complexity metric to predict -in real-time- sector complexity for trajectory-based operations (TBO). TBO will be implemented in the Next Generation Air Transportation System (NextGen). Trajectory-Based Complexity (TBX) is a modified aircraft count that can easily be computed and communicated in a TBO environment based upon predictions of aircraft and weather trajectories. TBX is scaled to aircraft count and represents an alternate and additional means to manage air traffic demand and capacity with more consideration of dynamic factors such as weather, aircraft equipage or predicted separation violations, as well as static factors such as sector size. We have developed and evaluated TBX in the Airspace Operations Laboratory (AOL) at the NASA Ames Research Center during human-in-the-loop studies of trajectory-based concepts since 2009. In this paper we will describe the TBX computation in detail and present the underlying algorithm. Next, we will describe the specific TBX used in an experiment at NASA's AOL. We will evaluate the performance of this metric using data collected during a controller-inthe- loop study on trajectory-based operations at different equipage levels. In this study controllers were prompted at regular intervals to rate their current workload on a numeric scale. When comparing this real-time workload rating to the TBX values predicted for these time periods we demonstrate that TBX is a better predictor of workload than aircraft count. Furthermore we demonstrate that TBX is well suited to be used for complexity management in TBO and can easily be adjusted to future operational concepts

Simulating Fleet Noise for Notional UAM Vehicles and Operations in New York

This paper presents the results of systems-level simulations using Metrosim that were conducted for notional Urban Air Mobility (UAM)-style vehicles analyzed for two different scenarios for New York (NY). UAM is an aviation industry term for passenger or cargo-carrying air transportation services, which are often automated, operating in an urban/city environment. UAM-style vehicles are expected to use vertical takeoff and landing with fixed wing cruise flight. Metrosim is a metroplex-wide route and airport planning tool that can also be used in standalone mode as a simulation tool. The scenarios described and reported in this paper were used to evaluate a fleet noise prediction capability for this tool. The work was a collaborative effort between the National Aeronautics and Space Administration (NASA), Intelligent Automation, Inc (IAI), and the Port Authority of New York and New Jersey (PANYNJ). One scenario was designed to represent an expanded air-taxi operation from existing helipads around Manhattan to the major New York airports. The other case represented a farther term vision case with commuters using personal air vehicles to hub locations just outside New York, with an air-taxi service running frequent connector trips to a few key locations inside Manhattan. For both scenarios, the trajectories created for the entire fleet were passed to the Aircraft Environmental Design Tool (AEDT) to generate Day-Night Level (DNL) noise contours for inspection. Without data for actual UAM vehicles available, surrogate AEDT empirical Noise-Power-Distance (NPD) tables used a similar sized current day helicopter as the Baseline, and a version of that same data linearly scaled as a first guess at possible UAM noise data. Details are provided for each of the two scenario configurations, and the output noise contours are presented for the Baseline and reduced noise DNL cases

The Influence of Mental Workload in Causes of System Degradation in Air Traffic Control

System safety and resilience is a critical concern in the air traffic domain. An important element of maintaining system safety and resilience is the ability of systems to degrade gracefully. However, previous research on the causes of system degradation in the air traffic domain are sporadic, and the potential interaction between the causes of degradation, and the resulting possible compound effect on the entire system, has been under-researched. An interview study was conducted with 12 retired controllers as participants. The results of a thematic analysis revealed the key causes of system degradation, and the associated impact on the ability of the controllers to prevent system degradation or recover the system. Findings have direct implications for identifying and mitigating potential risks of increasingly automated air traffic control systems

Predicting the Operational Acceptability of Route Advisories

NASA envisions a future Air Traffic Management system that allows safe, efficient growth in global operations, enabled by increasing levels of automation and autonomy. In a safety-critical system, the introduction of increasing automation and autonomy has to be done in stages, making human-system integrated concepts critical in the foreseeable future. One example where this is relevant is for tools that generate more efficient flight routings or reroute advisories. If these routes are not operationally acceptable, they will be rejected by human operators, and the associated benefits will not be realized. Operational acceptance is therefore required to enable the increased efficiency and reduced workload benefits associated with these tools. In this paper, the authors develop a predictor of operational acceptability for reroute advisories. Such a capability has applications in tools that identify more efficient routings around weather and congestion and that better meet airline preferences. The capability is based on applying data mining techniques to flight plan amendment data reported by the Federal Aviation Administration and data on requested reroutes collected from a field trial of the NASA developed Dynamic Weather Routes tool, which advised efficient route changes to American Airlines dispatchers in 2014. 10-Fold cross validation was used for feature, model and parameter selection, while nested cross validation was used to validate the model. The model performed well in predicting controller acceptance or rejection of a route change as indicated by chosen performance metrics. Features identified as relevant to controller acceptance included the historical usage of the advised route, the location of the maneuver start point relative to the boundaries of the airspace sector containing the maneuver start (the maneuver start sector), the reroute deviation from the original flight plan, and the demand level in the maneuver start sector. A random forest with forty trees was the best performing of the five models evaluated in this paper

User Selection Criteria of Airspace Designs in Flexible Airspace Management

A method for identifying global aerodynamic models from flight data in an efficient manner is explained and demonstrated. A novel experiment design technique was used to obtain dynamic flight data over a range of flight conditions with a single flight maneuver. Multivariate polynomials and polynomial splines were used with orthogonalization techniques and statistical modeling metrics to synthesize global nonlinear aerodynamic models directly and completely from flight data alone. Simulation data and flight data from a subscale twin-engine jet transport aircraft were used to demonstrate the techniques. Results showed that global multivariate nonlinear aerodynamic dependencies could be accurately identified using flight data from a single maneuver. Flight-derived global aerodynamic model structures, model parameter estimates, and associated uncertainties were provided for all six nondimensional force and moment coefficients for the test aircraft. These models were combined with a propulsion model identified from engine ground test data to produce a high-fidelity nonlinear flight simulation very efficiently. Prediction testing using a multi-axis maneuver showed that the identified global model accurately predicted aircraft responses

Identifying Functional Requirements for Flexible Airspace Management Concept Using Human-In-The-Loop Simulations

Flexible Airspace Management (FAM) is a mid- term Next Generation Air Transportation System (NextGen) concept that allows dynamic changes to airspace configurations to meet the changes in the traffic demand. A series of human-in-the-loop (HITL) studies have identified procedures and decision support requirements needed to implement FAM. This paper outlines a suggested FAM procedure and associated decision support functionality based on these HITL studies. A description of both the tools used to support the HITLs and the planned NextGen technologies available in the mid-term are presented and compared. The mid-term implementation of several NextGen capabilities, specifically, upgrades to the Traffic Management Unit (TMU), the initial release of an en route automation system, the deployment of a digital data communication system, a more flexible voice communications network, and the introduction of a tool envisioned to manage and coordinate networked ground systems can support the implementation of the FAM concept. Because of the variability in the overall deployment schedule of the mid-term NextGen capabilities, the dependency of the individual NextGen capabilities are examined to determine their impact on a mid-term implementation of FAM. A cursory review of the different technologies suggests that new functionality slated for the new en route automation system is a critical enabling technology for FAM, as well as the functionality to manage and coordinate networked ground systems. Upgrades to the TMU are less critical but important nonetheless for FAM to be fully realized. Flexible voice communications network and digital data communication system could allow more flexible FAM operations but they are not as essential

Ground-Ground Data Communication-Assisted Planning and Coordination: Shorter Verbal Communications

A human-in-the-loop simulation was conducted to investigate the operational feasibility, technical requirements, and potential improvement in airspace efficiency of adding a Multi-Sector Planner position. A subset of the data from that simulation is analyzed here to determine the impact, if any, of ground-ground data communication (Data Comm) on verbal communication and coordination for multi-sector air traffic management. The results suggest that the use of Data Comm significantly decreases the duration of individual verbal communications. The results also suggest that the use of Data Comm, as instantiated in the current simulation, does not obviate the need for accompanying voice calls

Detection of Operator Performance Breakdown as an Automation Triggering Mechanism

Performance breakdown (PB) has been anecdotally described as a state where the human operator "loses control of context" and "cannot maintain required task performance." Preventing such a decline in performance is critical to assure the safety and reliability of human-integrated systems, and therefore PB could be useful as a point at which automation can be applied to support human performance. However, PB has never been scientifically defined or empirically demonstrated. Moreover, there is no validated objective way of detecting such a state or the transition to that state. The purpose of this work is: 1) to empirically demonstrate a PB state, and 2) to develop an objective way of detecting such a state. This paper defines PB and proposes an objective method for its detection. A human-in-the-loop study was conducted: 1) to demonstrate PB by increasing workload until the subject reported being in a state of PB, and 2) to identify possible parameters of a detection method for objectively identifying the subjectively-reported PB point, and 3) to determine if the parameters are idiosyncratic to an individual/context or are more generally applicable. In the experiment, fifteen participants were asked to manage three concurrent tasks (one primary and two secondary) for 18 minutes. The difficulty of the primary task was manipulated over time to induce PB while the difficulty of the secondary tasks remained static. The participants' task performance data was collected. Three hypotheses were constructed: 1) increasing workload will induce subjectively-identified PB, 2) there exists criteria that identifies the threshold parameters that best matches the subjectively-identified PB point, and 3) the criteria for choosing the threshold parameters is consistent across individuals. The results show that increasing workload can induce subjectively-identified PB, although it might not be generalizable-only 12 out of 15 participants declared PB. The PB detection method based on signal detection analysis was applied to the performance data and the results showed that PB can be identified using the method, particularly when the values of the parameters for the detection method were calibrated individually